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Introduction 


In  a  preceding  Staff  Note  (Reference  1),  several 
possible  methods  for  constructing  a  priority  ranking  of 
multiple  attribute  alternatives  were  presented.  One  of  the 
methods  based  on  fuzzy  sets  (Reference  2)  was  used  to  form 
rankings  with  project  information  supplied  by  DLR,  on  a 
trial  basis.  This  note  provides  a  description  of  both  the 
method  and  the  trial. 

The  Need  for  Ranking  within  DLR 

DLR  essentially  maintains  a  "want  list"  (Long  Term 
Plan)  of  projects  of  various  types.  These  projects  are 
quite  diverse:  replacements  for  or  additions  to  existing 
land  forces  equipment,  upgrades  of  existing  equipment, 
requirements  for  acquistion  of  items  which  are  totally  new, 
et  cetera.  DLR's  justification  (through  Statements  of 
Requirement,  Program  Development  Proposals,  and  Program 
Change  Proposals)  of  the  projects  on  the  "want  list"  may 
have  a  major  impact  on  whether  they  eventually  receive 
funding  and  are  implemented. 

DLR  has  a  problem  in  that  there  is  neither  sufficient 
technical  manpower  available  to  allow  a  most  thorough 
justification  for  every  project,  nor  is  there  sufficient 
funds  for  every  project.  Thus,  a  priority  ranking  of  the 
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projects  would  provide  guidance  in  allocating  the  limited 
manpower  and  funds  available  to  those  projects  which,  in 
some  overall  sense,  are  most  desirable. 

Difficulties  Associated  with  Priority  Ranking 

One  of  the  main  difficulties  associated  with  creating 
a  priority  ranking  of  DLR  projects  is  that  they  are  not 
distinctly  comparable.  They  may  be  desirable  for  very 
different  reasons.  One  project  might  provide  for 
replacement  of  a  weapon  system  which  is  at  the  end  of  its 
life  cycle  and  is  becoming  expensive  to  maintain,  another 
project  might  provide  for  replacement  equipment  which  is 
safer  to  use  than  equipment  currently  used,  and  another 
project  might  contribute  to  the  fulfilling  of  an 
international  commitment  of  some  sort.  The  factors  or 
attributes  associated  with  the  projects  are  quite  varied. 

In  one  fashion  or  another,  either  explicitly  or  implicitly, 
a  trade-off  of  the  various  attributes  must  be  made  by 
decision  makers  in  order  to  produce  a  priority  ranking. 

Another  major  difficulty  associated  with  DLR's 
ranking  of  their  "want  list"  is  that  input  from  several 
sections  must  be  combined.  The  final  ranking  has  to  be  a 
consensus  of  contributing  individuals,  so  the  personal 
preferences  of  any  one  section  does  not  unduly  affect  the 
ranking . 


I 
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Basically,  DLR's  requirement  is  for  a  method  which 
constructs  a  priority  ranking  of  more  than  100  multiple 
attribute  alternatives,  based  on  inputs  from  5  or  more 
decision  makers.  It  was  desirable  that  DLR  have  a  ranking 
method  which  can  form  a  consensus  based  on  information 
submitted  independently. 

Overview  of  Ranking  Method 

The  implementation  of  a  prototype  ranking  method  is 
now  described  in  this  and  following  sections.  Every  project 
is  rated  by  decision  makers  on  a  number  of  attributes,  on  a 
scale  from  0,  which  is  least  favourable,  to  10,  which  is 
most  favourable.  These  ratings  must  be  consistent  with 
guidelines  specified  for  each  attribute.  If  an  individual 
is  not  certain  of  the  exact  value  for  a  specific  attribute 
rating,  he  can  use  an  interval  to  indicate  a  range  of 
acceptable  values.  For  instance,  if  he  could  not  decide 
amongst  the  rating  values  of  3,  4,  or  5,  he  could  choose  the 
interval  from  3  to  5.  The  individual  attribute  ratings  are 
subsequently  combined  into  consensus  attribute  ratings. 

Each  attribute  has  a  weight,  which  is  its  relative 
importance  compared  to  the  rest  of  the  attributes.  These 
weights  were  determined  from  pairwise  comparisons  supplied 
by  representatives  from  DLR  and  DLA.  From  the  comparisons, 
attribute  weights  were  derived  for  each  representative,  and 
these  are  averaged  for  use  in  the  ranking  procedure. 
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For  each  project,  the  consensus  attribute  ratings  are 
compared  with  the  corresponding  attribute  weights.  Two 
overall  ratings  based  on  the  results  of  the  comparison  are 
computed,  one  from  the  results  common  to  all  raters,  the 
other  from  those  common  to  at  least  half  the  raters.  Then, 
a  single  number  which  is  a  project's  figure  of  merit  is 
calculated.  (See  Figure  1.) 

The  figures  of  merit  for  the  projects  are  compared, 
and  an  overall  priority  ranking  is  made.  Those  projects 
which  have  figures  of  merit  that  are  close  together  are 
grouped  into  clusters,  to  indicate  that  they  are  essentially 
equal  in  rank. 

In  the  DLR  trial,  a  list  of  25  alternatives  were 
rated  on  10  attributes,  and  the  method  was  applied  to 
generate  a  priority  ranking.  The  results  presented  in  this 
note  were  generated  using  prototype  computer  routines 
written  in  APL.  Mr.  R.Y.  Lian  of  DMS  has*  since  written  a 
FORTRAN  program  which  can  be  used  to  implement  the  method. 
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FIGURE  1:  Derivation  of  a  Project  Figure  of  Merit 
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Attribute  Rating 

Each  of  the  25  projects  were  rated  according  to  the 
10  attributes  shown  in  Figure  2,  by  four  individuals  within 
DLR.  In  making  these  ratings,  they  used  the  attribute 
rating  scales  shown  in  Annex  A  as  a  guide.  Each  individual 
completed  a  sheet  similar  to  Figure  3  for  each  project, 
indicating  either  a  number  rating  or  an  interval  rating  (to 
accomodate  uncertainty)  for  each  attribute.  For  purposes  of 
further  analysis  on  the  computer,  these  ratings  were 
interpreted  as  pairs  of  numbers  (a  lower  bound  and  an  upper 
bound)  as  shown  in  Figure  4. 

A  computer  program  to  implement  the  ranking  method 
described  in  this  note  was  developed  by  Mr.  R.Y.  Lian  within 
DMS,  and  will  be  documented  separately.  Most  of  the  input 
to  the  computer  program  consists  of  contributing 
individuals'  ratings,  as  described  in  the  previous  para¬ 
graph.  For  instance,  in  the  trial  for  DLR,  4  individuals 
rating  each  of  10  attributes  on  25  projects  resulted  in 
4x10x25  =  1000  pairs  of  numbers. 

The  computer  program,  after  accepting  the  attribute 
rankings,  produces  consensus  ratings.  In  the  case  of  the 
trial  for  DLR,  four  pairs  of  numbers  representing  attribute 
ratings  for  a  particular  project  are  combined  to  produce  a 
consensus  rating  for  that  attribute  of  that  project. 
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External  Direction 
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Life  Expectancy 


Operational  Importance 


Safety 


Status  of  Staff  Action 


Urgency 


FIGURE  2  -  Attributes 
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An  example  of  the  formation  of  a  consensus  rating  is 
given  in  Figure  5,  with  rating  intervals  represented  by 
lines  of  X's.  A  consensus  rating  is  expressed  on  two 
levels.  A  100%  level  indicates  an  interval  which  is 
common  to  all  four  individuals.  (In  practice,  if  there  is 
no  such  common  agreement,  an  average  is  used.  Also,  if 
there  are  many  decision  makers,  then  any  interval  common  to, 
say,  80%  of  them  might  be  used  as  an  approximation  to  the 
100%  level.)  A  50%  level  indicates  an  interval  which  is 
common  to  at  least  half  of  the  individuals.  (Again,  in 
practice  if  there  is  no  such  common  area,  an  average  is 
used).  Also,  if  there  are  two  distinct  intervals  which  are 
common  to  at  least  half  of  the  individuals  they  are  joined 
to  form  one  long  interval.  If  need  be,  a  50%  interval  is 
extended  to  include  a  100%  level  which  was  calculated  as  an 
average,  since  the  ranking  method  assumes  the  100%  level  to 
be  contained  within  the  50%  level. 

An  example  showing  the  formation  of  consensus 
attribute  ratings  for  a  project  is  shown  in  Figure  6,  with 
intervals  specified  as  pairs  of  numbers. 
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FIGURE  5:  Formation  of  Consensus  Rating  for  an  Attribute 


P80676.PDF  [Page:  13  of  37] 


-  11  - 


Attribute 

- - 

Individual 

1  Rating 

Individual 

2  Rating 

Individual 

3  Rating 

Individual 

4  Rating 

Consensus 

100%  50% 

fp  Imp 

(6,8) 

(6,8) 

(5,7) 

(6,7) 

(6,7) 

(6,8) 

Urgency 

(5,5) 

(7,8) 

(5,6) 

(5,5) 

(5.75,5.75) 

(5,5.75) 

Int  Can 

(6,7) 

(6,7) 

(4,6) 

(6,6) 

(6,6) 

(6,7) 

Safety 

(1,1) 

(1,1) 

(1,1) 

(1,1) 

(1,1) 

(1,1) 

Avail 

(5,7) 

(4,6) 

(7,8) 

(6,7) 

(6.25,6.25) 

(5,7) 

Cost 

(7,7) 

(6. 5, 6. 5) 

(6,7) 

(6,6) 

(6. 5, 6. 5) 

(6,7) 

Ext  Dir 

(1,1) 

(1,1) 

(1,1) 

(1,1) 

(1,1) 

(1,1) 

Life  Exp 

(7,9) 

(9,9) 

(7,8) 

(8,9) 

(8.13,8.13) 

(7,9) 

Status 

(2,3) 

(2,2) 

(3,4) 

(2,2) 

(2. 5,2.5) 

(2,3) 

Ec  Nat 

(1,2) 

(5,5) 

(5,7) 

(1,1) 

(3.63,3.63) 
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Example:  The  Urgency  100%  interval  must  be  calculated  as  an  average: 

(5  +  7.5  +  5.5  +  5)  t  4  =  5.75.  The  Urgency  50%  interval 
comprises  any  overlap  corrmon  to  2  or  more  individuals  (the 
interval  (5,5) ,  and  is  extended  to  include  the  100%  interval, 
giving  (5,  5.75)  }  . 


FIGURE  6:  Example  of  Formation  of  Consensus 
Ratings  for  a  Hypothetical  Project 
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Attribute  Weights 

Although  the  ranking  method  developed  in  DMS  by  the 
author  allows  the  specification  of  attribute  weights  at  100% 
and  50%  levels,  in  the  DLR  trial  of  the  ranking  method 
specific  weights  were  generated  for  each  attribute.  The 
weights  were  formed  using  the  Analytic  Hierarchy  Process, 
described  in  Reference  3. 

A  number  of  representatives  from  DLR  and  one  from  DLA 
were  given  sheets  similar  to  FIGURE  7.  On  the  sheets  they 
indicated  preference  strengths  of  one  attribute  versus 
another,  for  each  pair  of  attributes.  For  instance  in 
Figure  7  a  check  mark  on  the  first  line  in  the  third  space 
would  indicate  that  an  individual  considers  "Operational 
Importance"  to  strongly  dominate  "Urgency",  when  judging  the 
various  projects.  (Philosophically,  the  attribute  weights 
are  considered  to  be  independent  from  the  specific  project 
to  be  ranked.  Practically,  the  representatives  filling  out 
the  preference  charts  probably  cannot  help  having  them  in 
mind . ) 


These  sheets  were  analyzed  according  to  methods 
presented  in  Reference  3  and  individual  sets  of  attribute 
weights  were  calculated.  These  weights  were  returned  to  the 
individuals  who  filled  out  the  sheets  for  feedback.  For  the 
most  part,  the  weights  were  considered  satisfactory.  Most 
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FIGURE  7 :  Pairwise  Comparison  Chart 


P80676.PDF  [Page:  15  of  37] 


P80676.PDF  [Page:  16  of  37] 


-  14 


suggested  amendments  were  minor,  with  a  few  large  (but  not 
unrealistic)  suggestions  for  amendment.  The  revised 
individual  weights  were  averaged  to  form  a  consensus  set  of 
weights,  shown  as  FIGURE  8. 

The  weights  shown  in  FIGURE  8  were  used  as  upper  and 
lower  interval  values,  at  both  the  100%  and  50%  levels,  as 
input  to  the  DMS  computer  program. 


T 
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FIGURE  8 


Attribute  Weights 
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As  illustrated  in  FIGURE  1,  for  each  project  the 
consensus  attribute  ratings  are  combined  with  the  attribute 
weights  in  order  to  produce  an  overall  rating 
representation.  The  manner  in  which  this  is  done  (adapted 
from  Reference  2)  is  described  in  Annex  B. 

The  overall  rating  representation  for  a  project 
consists  of  a  100%  level  interval,  and  a  50%  level  interval. 

A  figure  of  merit  for  the  project  (actually  a 
centroid  of  the  interval  representation)  is  calculated  as 

YlM  +  Y2A2 
A1  +  A2 

where  Y^  is  the  midpoint  of  100%  interval, 

Y2  is  the  midpoint  of  50%  interval, 

A1  is  the  area  of  100%  interval  (i.e.  [ . 5 ]  [ U—  L] 
where  "U"  is  upper  end  of  100%  interval  and 
"L”  is  lower  end  of  100%  interval), 

A2  is  the  area  of  50%  interval.  (i.e.  [.5][U-L] 

where  "U"  is  upper  end  of  50%  interval,  and 
"L"  is  lower  end  of  50%  interval.) 

In  FIGURE  9  overall  rating  representations  and  the  resulting 
figures  of  merit  are  shown  for  eight  hypothetical  projects. 
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(0,8) 

FIGURE  9  :  Derivation  of  Project 
Figure  of  Merit 
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Priority  Ranking 

The  computer  program  written  by  Mr.  R.Y.  Lian,  after 
accepting  individual  sets  of  ratings  and  consensus  attribute 
weights,  generates  figures  of  merit  for  the  various 
alternatives.  These  figures  of  merit  are  ordered  from 
highest  to  lowest  to  create  a  priority  ranking.  However, 
many  of  the  figures  of  merit  may  be  close  together  in  value, 
so  that  it  would  be  unrealistic  to  attempt  to  distinguish 
between  them.  So,  the  program  allows  the  user  to  specify  a 
level  of  "clustering",  where  projects  which  are  closest 
together  in  value  are  essentially  considered  equal.  The 
program  also  automatically  generates  some  "clustered" 
answers  at  different  levels  to  allow  comparison  with  the 
user  specified  one. 

FIGURE  10  illustrates  figures  of  merit  that  were 
produced  from  DLR  trial  data,  with  the  projects  identified 
simply  as  1  through  25,  along  with  a  sample  clustering. 


CONCLUSION 


It  must  be  emphasized  that  a  structured  priority 
ranking  method  for  multiple  attribute  alternatives,  on 
complicated  scenarios  such  as  the  one  DLR  has,  provides  a 
first-cut  answer.  Consideration  of  "special  cases"  and 
fine-tuning  must  be  left  to  decision  -  makers,  not  to  the 
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AMENDED 

FIGURE  OF  MERIT 

PRIORITIZED 

PROJECT  LIST 

(CLUSTERED  TO  9  GROUPS) 

6.02 

4,6 

5.73 

5,7 

5.17 

18,19 

4.91 

1,8,9 

4.71 

3,16,17,21,23 

4.47 

13,22 

4.13 

2,15,20 

3.53 

10,11,12,24,25 

2.86 

14 

FIGURE 

OF  MERIT 

PRIORITIZED 

PROJECT  LIST 

(NO  CLUSTERING) 

6.09 

6 

5.96 

4 

5.81 

5 

5.66 

7 

5.19 

18 

5.14 

19 

4.95 

1 

4.88 

8 

4.85 

9 

4.75 

3 

4.71 

21 

4.70 

16 

4.68 

23 

4.65 

17 

4.50 

22 

4.44 

13 

4.18 

2 

4.11 

15 

4.06 

20 

3.65 

12 

3.56 

25 

3.56 

10 

3.56 

24 

3.45 

11 

2.86 

14 

FIGURE  10 :  Sample  Clustered  Ranking 
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computer.  The  value  of  the  computer  produced  ranking  is  only 
in  providing  a  reasonable  starting  configuration,  which  can 
be  further  improved  by  human  decision  making. 


T 


T 
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Annex  A 

To  ORAE  Staff  Note  3/83 
Dated  September  1983 

Attribute  Rating  Guidelines 

(When  using  the  following  scales,  a  decision  maker  will,  if 
appropriate,  choose  a  rating  value  which  is  between  two  of 
the  described  values.  Also,  the  ranking  method  treats 
differences  of  a  certain  amount  as  equal,  no  matter  where 
they  occur  on  a  scale.  So,  for  instance,  the  difference 
between  1  and  3  on  a  given  scale  has  the  same  value  as  the 
difference  between  7  and  9  on  the  same  scale.) 

Operational  Importance: 

1  -  Insignificant  contribution  to  operational  capability 

2  -  Partial  contribution  to  desirable  capability 

3  -  Desirable 

4  -  Partial  contribution  to  necessary  capability 

5  -  Necessary  capability 

7  -  Partial  contribution  to  essential  capability 

Essential  capability 
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Urgency : 

1  -  Implementation  date  at  least  two  years  beyond  minimum 
DP MS  staffing  milestones  and/or  implementation  could 
be  deferred  two  or  more  years 

3  -  At  optimum  time  in  relation  to  DPMS  milestones  and  up 
to  two  years  flexibility  in  implementation  date 

5  -  At  minimum  leadtime  in  relation  to  DPMS  milestone  and 
less  than  two  ears  flexibility  in  implementation  date 

7  -  Accelerated  staffing  required,  or  at  minimum  leadtime 
and  no  flexibility  in  implementation 

9  -  Immediate  essential  requirement 

Cost : 

1  -  $1,000  million 

2  -  $600  to  $1,000  million 

3  -  $300  to  $600  million 

4  -  $100  to  $300  million 

5  -  $45  to  $100  million 

6  -  $30  to  $45  million 

7  -  $15  to  $30  million 

8  -  $5  to  $15  million 

9  -  Less  than  $5 


million 
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Availability: 

1  -  Almost  certain  to  have  major  problems  -  cost  changes, 
development  time,  production,  coming  into  service, 
delivery 

3  -  50%  or  more  chance  of  significant  availability 

problems 

5  -  25%  -  50%  chance  of  significant  availability  problems 

7  -  low  risk 

9  -  almost  no  risk 

External  (to  DND)  Direction: 

1  -  No  specific  external  direction 

5  -  Specific  equipment  also  serves  DND  low  operational 
requirement 

7  -  DND  committed 

9  -  Specific  equipment  also  serves  DND  high  operational 
requirement 

Life  Expectancy: 

1  -  Can  be  used  almost  indefinitely  (eg.  helmets) 

3  -  Can  be  extended  very  cost  effectively 

5  -  Extension  moderately  cost  effective 

7  -  Extension  marginally  cost  effective 

Finite  and  no  flexibility 
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Status  of  Staff  Action: 

1  -  No  formal  status 

2  -  DCDS  only  approval  in  DSP  period  or  in  LTP 

3  -  HC"  Capital 

4  -  "C"  Capital  and  PCP  milestone  achievable  within  DPMS 

lead-times;  or  PCP  achievable  and  DCDS  has  approved 
PCP  staff  action  in  lieu  of  PDP 

5  -  PCP  available  for  final  circulation  and  sign-off 

within  six  months  (which  supports  target 
implementation  date) 

6  -  PCP  on  final  circulation  and  tentative 

committee (appropriate  approval  level)  scheduled 

7  -  PCP  approved  or  expected  within  two  months 

8  -  "B"  Capital  and  TBS  milestone  firm 

9  -  "A”  Capital 

Economic  Nationalism: 

1  -  Directed  to  Canadian  source  and  lengthy  or  high  risk 
industry  development  involved 
3  -  Uncertain  as  to  impact 

5  -  Canadian  source  possible  or  definite  with  significant 

cost  but  low  risk 

7  -  Canadian  source  possible  or  definite  and  (in  full  or 
part)  presents  no  significant  additional  cost  or  risk 
9  -  No  delays,  cost,  or  risk  related  to 
Canadian/offshore  procurement 


▼ 


T 
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Safety: 

0  -  No  specific  safety  implications 

2  -  Corrects  a  condition  likely  to  incur  bodily  injury 

4  -  Corrects  a  condition  likely  to  incur  permanent 

disability  or  a  slight  risk  to  life 
6  -  Corrects  a  condition  with  a  moderate  risk  to  life 

8  -  Corrects  a  condition  with  a  high  risk  to  life 

9  -  Corrects  a  condition  with  a  very  high  risk  to  life  on 

a  large  scale 

International  Commitment: 

0  -  No  commitment  exists 

2  -  Satisfies  partially  a  low-level  or  moderate 

commitment 

4  -  Satisfies  completely  a  moderate  commitment 

6  -  Partially  satisfies  a  NATO  Goal  or  Pri  II  or  III 

7  -  Partially  satisfied  a  high-level  ( MND  or  Gov't) 

commitment,  or  NATO  Goal  of  Pri  I 

8  -  Satisfies  a  NATO  Goal  of  Pri  I 
Satisfies  a  high-level  (MND  or  Gov't)  commitment. 
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Annex  B 

To  ORAE  Staff  Note  3/83 
Dated  September  1983 

CONSTRUCTION  OF  OVERALL  RATING  REPRESENTATIONS 
IN  THE  FUZZY  SET  METHOD 

The  construction  of  an  overall  rating  representation 
is  explained  here  by  means  of  an  example.  The  calculation 
of  the  overall  100%  level  upper  and  lower  endpoints  for  an 
alternative  with  5  attributes,  a  to  e,  is  shown. 

Calculation  of  the  50%  level  endpoints  would  be  done  in 
similar  fashion. 

The  100%  level  intervals  for  attribute  weights  and 
ratings  are  reproduced  below: 


| Weights  |  100%  |  (.2, .4)  (.2, .4)  C . 1 , . 3 )  (0,.l)  (.1,.2)  [ 
(Ratings  |  100%  |  (2,4)  (8,10)  (3,5)  (8,10)  (7,9)  ( 


The  upper  endpoint  of  the  overall  rating  at  the  100% 
level  is  an  expression  of  the  form: 


5 


5 

X  wj 
j  =  l 
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where  rj  is  the  upper  endpoint  of  the  jth  attribute  rating 
at  the  100%  level,  and  wj  is  either  the  lower  or  upper 
endpoint  of  the  jth  weight  at  the  100%  level.  The  wj  values 
are  chosen  so  as  to  maximize  r,  and  are  determined  with  a 
set  of  computations  which  are  explained  presently. 

A  series  of  calculations  at  most  equal  to  the  number 
of  attributes  is  performed.  In  the  first  one,  r  is 
calculated  with  w^  being  the  upper  endpoint  for  k 
corresponding  to  a  highest  rating  value  rkf  and  all  other  wj 
values  (j  ^  k)  as  lower  endpoints.  In  the  second 
calculation,  r  is  worked  out  with  w|<  and  wi  as  upper 
endpoints  for  those  values  of  k  and  i  which  correspond  to 
the  two  highest  rating  values  r^  and  ri  and  all  other  wj 
values  (j  k  or  i)  are  lower  endpoints.  This  pattern 
continues,  with  successive  wj  values  for  the  third— highest , 
f ourth-highest ,  etc.  rj  values  being  switched  from  the  lower 
to  the  higher  endpoint.  It  can  be  shown  (Appendix  1)  that 
the  maximum  r  value  calculated  amongst  such  successive 
trials  is  indeed  the  maximum  r  value  that  can  be  generated 
by  choosing  any  combination  of  wj  and  rj  values  which  are 
within  the  specified  intevals.  The  calculation  results  are 
shown  in  Table  II. 


T 


T~T 
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TABLE  II 

MAXIMUM  r  CALCULATIONS 


TRIAL 

W1 

H 

w 

2 

r2 

W3 

r3 

D 

W5 

r5 

r 

1 

.2 

4 

.4 

10 

.1 

5 

0 

10 

.1 

9 

7.8 

2 

.2 

4 

.4 

10 

.1 

5 

.1 

10 

.1 

9 

8.0 

3 

.2 

4 

.4 

10 

.1 

5 

.1 

10 

.2 

9 

8.1 

4 

.2 

4 

.4 

10 

.3 

5 

.1 

10 

.2 

9 

7.6 

5 

.4 

4 

.4 

10 

.3 

5 

.1 

10 

.2 

9 

7.1 

Maximum  r  value  is  8.1 


TABLE  III 

MINIMUM  r  CALCULATIONS 


TRIAL 

W1 

D 

W2 

r2 

w 

3 

r3 

H 

w 

5 

r5 

r 

1 

.4 

2 

.2 

8 

.1 

3 

0 

8 

.1 

7 

4.3 

2 

.4 

2 

.2 

8 

.3 

3 

0 

8 

.1 

7 

4.0 

3 

.4 

2 

.2 

8 

.3 

3 

0 

8 

.2 

7 

4.3 

4 

.4 

2 

.4 

8 

.3 

3 

0 

8 

.2 

7 

4.8 

5 

.4 

2 

.4 

8 

.3 

3 

.1 

8 

.2 

7 

5.1 

Minimum  r  value  is  4.0 
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In  calculating  the  lower  endpoint  of  the  overall 
rating  at  the  100%  level,  the  same  expression  for  r  is  used, 
but  the  rj  values  are  the  lower  endpoints  of  attribute 
ratings,  and  wj  values  are  chosen  so  as  to  minimize  r. 

Again,  a  number  of  calculations  most  equal  to  the  number 

of  attributes  are  performed.  In  the  first  one,  r  is 
calculated  with  wj^  being  the  upper  endpoint  for  k 
corresponding  to  a  lowest  rating  value  r^,  and  all  other 
wj  values  (j  =*=  k)  as  lower  endpoints.  In  the  second 
calculation,  r  is  worked  out  with  w^  and  wi  as  upper 
endpoints  for  those  values  of  k  and  i  which  correspond  to 
the  two  lowest  rating  values  r^  and  rj_,  and  all  other  wj 
values  (j  k  or  i)  are  lower  endpoints.  This  pattern 
continues,  with  successive  wj  values  for  third-lowest, 
fourth-lowest ,  etc.  rj  values  being  switched  from  the  lower 
to  the  higher  endpoint.  It  can  be  shown  that  the  minimum  r 
value  calculated  amongst  such  successive  trials  is  indeed 
the  minimum  r  value  that  can  be  generated  by  choosing  any 
combination  of  Wj  and  rj  values  within  the  specified 
intervals.  The  calculation  results  are  shown  in  Table  III. 

So,  the  overall  rating  at  the  100%  level  is  the 
interval  (4.0,  8.1).  Interval  calculations  at  the  50%  level 
are  done  in  the  same  way  as  the  preceding  presentation, 
using  the  50%  level  (weight  and  rating)  upper  and  lower 
endpoints . 


T 


n 
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APPENDIX  1 
TO  ANNEX  B 


Hypothesis 


The  maximum  value  for 


7  =  iri 


T  w  .r . 
X  3  3 


n 

z  ^ 


j=l 


3 


(where  the  alternative  values  are  generated  by  choosing  either  a 


"low"  or  a  "high"  value  for  each  w.) 

3 

can  be  formed  by  using  "high"  w^  values  for  a  set  of  r^  values  which  are  all 

greater  than  or  equal  to  any  r^  value  paired  with  a  "low"  w^ . 

m-1  n 

Z  w . r .  +  Z  w  .r  , 

..  .  1-1  3  3  i=m  3  3 

i .  e .  Maximum  =  J u - 


n 


w 


j=l 


j 


for  some  m. 


where  r  >  r,  for  r  e  ri  ,  r?  ,  .  . .  ,  r  , 
a  —  b  a  1  ^  m-1 


and  r,  e 
b 


r  r  .  . . .  r 

m,  ml,  n 


and  w j ,  1  <  j  <  m-1 ,  are  "high"  values 


and  w . ,  m  <  j  <  n,  are  "low"  values 
3  ~ 
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Proof  by  Contradiction 


Assume  max  r 


-  ?Vi 


occurs  with  r^  >  rd  such  that  takes  the  low  value  w^  ,  and  wd  the  high 


value  of  w^  .  Then  by  this  assumption; 
a 


(a)  If  we  use  both,  low  values  for  w^  and  wd» 


t  Z 

i.e.  w  and  w  ,  then  —  — 

u  d  rLZ  max 


(b)  If  we  use  both  high  values  for  w^  and  wd? 


i.e.  w*  and  wb,  then  r^  < 


h  Z  .  , 

Let  w  =  w  +  b 
u  u  u 


h  Z  ,  , 
Wd  =  Wd  +  bd 


b  ,  b .  >  0 
.  u  d 


z.  z 

and  letZw.  and  Iw.r.  imply  the  use  of  vr  and  w 
3  j  J  u  a 


Then  for  (a) 


Ivi  *  Vd  :  ^V.i 

+  6d 


i 


T 
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Cross  multiplying  and  cancelling  as  appropriate,  we  get 


r^Zw.  >  2w.r. 
<13  3  3 


and  for  (b) 


Zw  r,  +  b  ,r ,  7w,r ,  +  b  r  +  b  r 

-ii  dd^^Ti  uu  dd 


Iw.  +  bd 


Tw.  +  b  +  bJ 
^3  u  d 


Cross  multiplying  gives: 


Z"JZV3  +  VdZ"j  +  buZVj  +  bubdrd  +  bdZwjrj  +  bJrd 


>  Zw.  Zw.r.  +  b  r  Zw.  +  b.r.Zw.  +  b  Iw  r  +  b  b  r  +  b*r 

3  33  uu  3  d  d  3  d  3D  udu  dd 


b  Zw.r.  +  b  bJrJ  >  b  r  Zw.  +  b  b  r 
u  3  3  udd  uu  3  udu 


Tw  r  +br  >r  Zw.  +  b  r 
^  j  j  dd  u  3  d  u 


Now  from  the  result  for  case  (a) ,  we  can  replace  Zw^r^  in  this  inequality 

by  r.  Zw .  .  Hence 
d  3 


rd  (Zwj+  0  >ru  (z"j  +bd) 


•  -  r ,  >  r 
d  u 

which  contradicts  the  assumption.  Hence  the  hypothesis  is  proved.  The  proof 
for  the  minimum  r  proceeds  similarly. 
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